Skip to content

cl/beacon: report pending blob backfill availability - #23213

Draft
domiwei wants to merge 3 commits into
kewei/fix-blob-backfill-boundaryfrom
kewei/blob-backfill-availability
Draft

cl/beacon: report pending blob backfill availability#23213
domiwei wants to merge 3 commits into
kewei/fix-blob-backfill-boundaryfrom
kewei/blob-backfill-availability

Conversation

@domiwei

@domiwei domiwei commented Aug 12, 2026

Copy link
Copy Markdown
Member

Summary

  • return a retryable JSON 503 when requested canonical blob data is still pending backfill instead of returning a misleading empty success
  • bound blob-sidecar retries and prevent historical backfill from starving live sync
  • track completed canonical coverage as precise Deneb/Fulu ranges and preserve partial progress across retries
  • accept legal partial or reordered blob_sidecars_by_root responses, verify them by identity and proof, and retry only missing sidecars
  • make Deneb and Fulu recovery failures truthful and repair stale Fulu recovery state from actual persisted sidecars
  • invalidate completion around canonical changes and harden persisted blob/API availability state

Relationship to #23138

This is a stacked follow-up to the minimal #23138 fix. #23138 contains only the exclusive/moving FrozenBlobs() boundary and the 16-peer startup blocker from #22429.

All retry, concurrency, partial-response, completion, reorg, Fulu, storage, and 503 availability behavior is intentionally reviewed here as one lifecycle change. Review comments originally anchored to the earlier combined #23138 diff are addressed on this branch when their code moved here.

The temporary base is kewei/fix-blob-backfill-boundary. After #23138 merges, this PR can be retargeted to main without changing its effective diff.

API compatibility

The Beacon API endpoint schemas do not explicitly enumerate 503 for these blob endpoints. This PR intentionally uses it as an Erigon extension for temporary backfill unavailability. The response follows the shared JSON error shape (code and message) and uses Content-Type: application/json.

Available requested indices or versioned hashes remain a 200. Empty blocks, unmatched filters, side branches, frozen data, and completed or disabled backfill preserve their normal behavior.

Validation

  • affected package suites for Beacon handlers, PeerDAS, blob storage, downloader/network, forkchoice/stages, and RPC
  • focused concurrency and reorg tests under -race, including repeated PeerDAS coalescing and canonical-transition schedules
  • make lint (final complete pass: 0 issues twice)
  • git diff --check
  • fresh Standards and Spec review of the final cumulative diff
  • two independent final Adversarial/Lifecycle reviewers converged with no actionable Critical/High/Medium/Low finding

Review disposition

The feedback originally anchored to the earlier combined #23138 diff was applied here after the split. It covers legal sparse blob responses, truthful Deneb/Fulu completion, bounded live/backfill admission, canonical reorg and commit visibility, PeerDAS cancellation/coalescing, partial persisted data, snapshot/storage transitions, query validation, and test lifecycle hygiene.

Broader pre-existing hardening, including a separate blob_sidecars_by_range admission gate and crash-orphan cleanup, is intentionally outside this PR.

@domiwei
domiwei force-pushed the kewei/blob-backfill-availability branch from 5d3af81 to 6e5df70 Compare August 12, 2026 16:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant